Associations between text and imagery are currently performed almost exclusively using manual methods. With text primarily in the hard copy, as opposed to electronic form, humans are generally required to read, understand and associate text with images in practically all cases.
The commercial knowledge management industry and the national intelligence community have focused on the research and development of tools to correlate and combine qualitative text data using words, phrases and concepts as the basis to search, correlate, combine and abstract from the corpus of electronic texts. Information operations, especially in the symbolic and cognitive domains, require the ability to combine and model structured and unstructured text data across multiple languages.
The DARPA Dynamic Multiuser Information Fusion (DMIF) program developed message parsing capabilities to convert and extract quantitative data sets (target vectors) from structured tactical report formats. The U.S. DoD Joint Directors of Laboratories (JDL) Data Fusion Subpanel has developed a three-level model which characterizes the capabilities of data fusion technologies. Commercial tools developed by Excalibur and Autonomy are pioneering the manipulation of unstructured text, audio and video data to perform fusion functions that approach those defined in the JDL fusion model, including level 1 fusion of words, topics and concepts.
Data fusion developers must consider approaches to perform fusion of both qualitative and quantitative data to develop understandings of situations in which both categories of data are available. Combined fusion processes (FIG. 1) will allow sense data (quantitative) and source data (most often qualitative) to be combined to provide a complete understanding of complex problems.
Knowledgeable subject area analysts currently tackle such problems, but the increasing deluge of global qualitative and quantitative data makes it difficult for those analysts to consider and assess all available data. Combined qualitative-quantitative data fusion and mining technologies will allow all available data to be related and analyzed to bring to the human analysts the most relevant 3-domain model implications, and to allow the analysts to drill-down to the most significant supporting data.
In the current environment, however, with on-line news services and mega-information services available via the Internet, people are unable to keep up with the large volume of unstructured electronic information becoming available. Manual methods are too slow and involve too many interactions in a time of scarce human resources.
One proposed solution to this problem uses metadata; namely key words and computed indices used to label each image as a whole. While such techniques can be used to locate images for some applications, metadata associations still require human interaction, and are similarly too slow. The need remains, therefore, for a system and methodology that automatically correlates textual references to geographic locations including imagery representative of such locations, preferably through contextual inferences as opposed to key word searching.